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ELECTRONIC NEURAL NETWORKS 

JET PROPULSION LABORATORY 

BACKGROUND 


• NEURAL NETWORKS OFFER A TOTALLY NEW APPROACH TO INFORMATION 
PROCESSING THAT IS ROBUST, FAULT-TOLERANT, AND FAST. 

• MODELS OF NEURAL NETWORKS ARE BASED ON HIGHLY PARALLEL AND 
DISTRIBUTIVE ARCHITECTURES. 

• JPL IS DEVELOPING ELECTRONIC IMPLEMENTATIONS OF ARTIFICIAL NEURAL 
NETWORK S TO EXPLOIT THEIR EMERGENT PROPERTIES FOR "INTELLIGENT 
KNOWLEDGE ENGINEERING APPLICATIONS. 


• IN AN ELECTRONIC EMBODIMENT, "NEURONS" ARE REPRESENTED BY 

THRESHOLD AMPLIFIERS, AND "SYNAPSES" BY RESISTORS. INFORMATION IS 
STORED IN THE VARYING STRENGTHS OF SYNAPTIC CONNECTIONS. 


• ELECTRONIC IMPLEMENTATIONS OF APPLICATION - SPECIFIC HIGH SPEED 
NEUROPROCESSORS FOR: 


• ASSOCIATIVE RECONSTRUCTION 

• CARTOGRAPHIC ANALYSIS 

• RESOURCE ALLOCATION 

• GLOBAL OPTIMIZATION 

• AUTONOMOUS CONTROL 
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INTRODUCTION 


• ARTIFICIAL NEURAL NETWORKS : 


MASSWELT PARALLEL ARCHITECTURES INSPIRED 
BY THE NATURE'S APPROACH TO INTELLIGENT 
INFORMATION PROCESSING 


• KEY QUESTIONS : 


WHAT GOOD ARE THEY ? 

THEY DO SOMETHING UNIQUE FOR SPACE 
olATlON ? 


WHEN WILL THEY DELIVER ? 
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NUMBERS ! 


10 14 

A HUMAN BRAIN HAS OVER 10 NEURONS AND 10 SYNAPSES. 

3 

HARDWARE IMPLEMENTATIONS TODAY ARE AT ABOUT 10 NEURONS 
5 

AND 10 SYNAPSES. 


BUT • THE BRAIN IS NOT FULLY CONNECTED ! 

. THE CURRENT ARTIFICIAL NEURAL NETWORKS ARE ALREADY 

PROVING THEMSELVES EXTREMELY USEFUL. 

. NEURAL NETWORKS ARE NOT GENERAL PURPOSE COMPUTERS. 

THEY ARE SPECIAL-PURPOSE HIGH PERFORMANCE 
CO-PROCESSORS. 
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JPL NEURAL NETWORK 


BASIC COMPONENTS: NEURONS 

SYNAPSES 

ELECTRONIC IMPLEMENTATION: 



INPUT 


NEURON 


OUTPUT 
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ELECTRONIC NEURAL NETWORKS 

JET PROPULSION LABORATORY 


NEURAL NETWORK ARCHITECTURES 
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ELECTRONIC IMPLEMENTATIONS 


° JPL'S APPROACH HAS BEEN: 

° TO DEVELOP NEURAL NET "BUILDING BLOCKS" FOR MODULAR HARDWARE 
TO VERIFY DESIGN O F NEUROCIRCUITS IN SIMULATION 

TO TAILOR "NEUROPROCESSORS" FOR SELECTED APPLICATIONS 
° TECHNOLOGIES: 


VLSI: CMOS, BIPOLAR, AND EEPROM 

(FLOATING GATE) 


THIN FILM: AMORPHOUS SEMICONDUCTORS, ELECTROCHROMIC 
MATERIALS, AND CHEMICAL MICROSWITCHES 


HYBRID: VLSI/THIN FILM, SINGLE/MULTI-CHIP, WAFER-LEVEL INTEGRATION 
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VLSI/THIN FILM HYBRID HARDWARE 
FOR NEUROCOMPUTING 


1024 CAPACITOR-REFRESH, 
ANALOG SYNAPSE CHIP 



32 x 32 BINARY SYNAPSE CHIP 



17-NEURON ARRAY CHIP 
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ELECTRONIC NEURAL NETWORKS 

JET PROPULSION LABORATORY 

CASCADABLE, PROGRAMMABLE, 32 x 32 SYNAPTIC CMOS CHIP 
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FROM NEURON OUTPUTS 



AU • M 


• A BUILDING BLOCK FOR VLSI NEURAL 
NET HARDWARE 

• PROVIDES OVER 10 9 ANALOG 
OPERATIONS/SEC 

• POTENTIAL APPLICATIONS: 

— MULTI SENSOR DATA FUSION 

— ASSOCIATIVE RECONSTRUCTION 

— PATTERN RECOGNITION 
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JPL COMPUTATION WITH ANALOG 

PARALLEL PROCESSING 


• HIGH-DENSITY 
INTERCONNECTIONS 

• NON-VOLATILE 
ANALOG MEMORIES 


NEURAL NETWORK 
ARCHITECTURES 


CELLULAR ARRAY 
PROCESSORS 

• ASSOCIATIVE 

• PATH PLANNING 

MEMORY 



• RESOURCE 

• AUTONOMOUS 

ALLOCATION 


CONTROL 

• GLOBAL 
OPTIMIZATION 

• CARTOGRAPHIC 
ANALYSIS 


CUSTOM THIN FILM 
MICRODEVICES 
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FEATURES OF NEUROPROCESSORS 


• APPLICATION-SPECIFIC ARCHITECTURES 

processIng^ massively parallel, analog, asynchronous 


• EXTREMELY HIGH SPEED : TERRA-OPS RANGE 

• INHERENT FAULT-TOLERANCE 


self-organ^e IL,TIES T ° learn from experience AND 


SPEED DISTAL* MACHINES' C0MPLEMENTING THE ABILITIES OF HIGH 


APPLICATIONS OF NEUROPROCESSORS 


. AUTOMATION AND ROBOTICS 


. ON-BOARD, REAL-TIME , GLOBAL OPTIMIZATION 

ALLOCATION AND MANAGEMENT OF RESOURCES 

cn 

to 

• AUTONOMOUS, SCHEDULING, SEQUENCING, AND EVENT-DRIVEN 
MISSION REPLANNING 


. SCIENCE DATA ANALYSIS AND MANAGEMENT 

. PATTERN RECOGNITION, CLASSIFICATION 

. MODELING OF LARGE SYSTEMS 

CODING, DECODING, ASSOCIATIVE RECONSTRUCTION 
FROM CORRUPT DATA 
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__ NEURAL NETWORK HARDWARE FOR 
JPL terrain trafficability DETERMINATION 





155 


•' ) 1 1 » ! 1 ‘ .1 1 ( ! } 


A DEDICATED PROCESSOR FOR PATH PLANNING 


PATH COMPUTATION 


• MASSIVELY PARALLEL, ANALOG 
ASYNCHRONOUS PROCESSING 


• ACCEPTABLE SOLUTION IN REAL TIME 

THAN THE BEST SOLUTION AFTER A LONG 
TIME 


• ORDERS OF MAGNITUDE SPEED 

™Sts ARTICULARLY for " what 
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HARDWARE DETAIL FOR SIGNAL SORTER 
AND MAP SEPARATES APPLICATIONS 
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JPL NEURAL NETWORK SYSTEM INTERFACE 


DIRECT 

INPUT 


DIRECT 
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ini ELECTRONIC NEURAL NETWORKS 

RAPID HARDWARE PROTOTYPING OF 
NEURAL PROCESSORS FOR "REAL" PROBLEMS 


THE PROBLEMS 

• MAP KNOWLEDGE BASE APPLICATIONS REQUIRING LARGE AMOUNTS 
OF DATA AND HIGH SPEED PROCESSING 

• CROSS-COUNTRY MOBILITY DETERMINATION: A MULTIDIMEN- 

SIONAL EUCLIDEAN DISTANCE MINIMIZATION PROBLEM 


• GENERATION OF MAP SEPARATES: AN IMAGE SEGMENTATION 

PROBLEM 


•DETERMINATION OF "BEST" PATH: A ROUTING PROBLEM 


THE SOLUTION 

• DEDICATED NEUROPROCESSORS IMPLEMENTING NEURAL ALGORITHMS 
FOR SOLVING THESE PROBLEMS 

• THESE NEUROPROCESSORS WILL BE INTERFACED WITH A PORTABLE 
ASAS WORK STATION (PAWS) FOR USER EVALUATION AT HIE CENIER 
FOR SIGNALS WARFARE (CSW) 


159 


) * 1 1 1 1 J j ‘ ^1 "J ' \ ' f i • } 

JPL NEURAL NETS FOR ROBOTIC CONTROL 


LAYERED, FEED-FORWARD NETS WITH ABILITIES TO 'LEARN' FROM 'EXPERIENCE' 



n l n2 03- n 0 



• LEARNING ALGORITHM: ERROR BACK PROPAGATION 

• 'KNOWLEDGE' IS ACCUMULATED IN THE ANALOG SYNAPTIC WEIGHTS 
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ERROR BACKPROPAGATION ALGORITHM 

FOR LEARNING 


TRAINING SET 


RANDOMIZE SYNAPTIC WEIGHTS 

I 

> FEED INPUT < 

I 

FORWARD PASS 

I 


COMPARE OUTPUT WITH 
DESIRED TARGET 


YES < 

I 

STORE WEIGHTS 




GENERATE ERROR VECTOR 

I 

IS LEARNING COMPLETE? 

I 

ERROR < TOLERANCE - 


ADJUST WEIGHTS 


NO 


) I 


) I \ I 
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JPL RESOURCE ALLOCATION MATRIX 
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JPL 


RESOURCE ALLOCATION PROCESSOR 

SIMULATION 
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GLOBAL OPTIMIZATION NEUROPROCESSOR 


• RESOURCE ALLOCATION 

• DYNAMIC ASSIGNMENT 

• MESSAGE ROUTING 

• TARGET-WEAPON PAIRING 

• LOAD BALANCING 

• MULTI-TARGET TRACKING 

• SEQUENCING / SCHEDULING 

• REAL TIME, ADAPTIVE 
MISSION RE-PLANNING 



MINI: SUES 
3 CUNCENIHAIUIIS 


Al I Ml 3 SUES III ACM 

CUNCENI.VAIOH IN achieve 
MINIMUM irEHAl.l. COS I 
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NEUROPROCESSING APPROACH OFFERS OVER 4 ORDERS OF 
COMPUTING TECHNIQUES NCEWIENT ° VER ™ E C0NVENT| 0 NAL 


• • 


WH?ph R arp mot eaom JO MULTIPLE ASSIGNMENTS ARE POSSIBLE 
TBCHNIQUES ° T EAS LY ACC °MPLISHED BY CONVENTIONAL 
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ELECTRONIC NEURAL NETWORKS 

JET PROPULSION LABORATORY 

PROGRAMMABLE, NONVOLATILE, HIGH-DENSITY, THIN FILM 
SYNAPTIC ARRAY FOR INFORMATION STORAGE 


• POTENTIAL FOR HIGH DENSITY 
APPROACHING 10 9 BITS/CM 2 

• FAULT-TOLERANCE 

• ASSOCIATIVE, CONTENT-ADDRESSABLE 
10 /<m RECALL 

*CT * N0 moving PARTS 

» • STORAGE IN “PASSIVE,” RAD-HARD 

jy INTERCONNECTIONS 

f 

**imn** 
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ELECTRICALLY PROGRAMMABLE 
READ ONLY THIN-FILM SYNAPTIC ARRAY 



4-fim FEATURE SIZE 
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CONCLUSIONS 


. THE FIELD OF NEUROPROCESSING HAS SIGNIFICANTLY ADVANCED IN RECENT 
YEARS. INVESTMENTS OF DOD AND NASA ARE RESULTING IN THE 
DEVELOPMENT OF APPLICATION-SPECIFIC, HIGH PERFORMANCE, MODULAR 
NEUROPROCESSORS. 

. SUCH NEUROPROCESSORS OFFER TOTALLY NEW CAPABILITIES, 
COMPUTATIONAL BREAKTHROUGHS, AND ORDERS OF MAGNITUDE 
PERFORMANCE ENHANCEMENT WHERE CONVENTIONAL PROCESSING METHODS 

CHOKE. 

. DOD IS MOVING AHEAD IN THE DIRECTION OF HARDWARE PROTOTYPING OF 
DEPLOYABLE NEUROPROCESSORS FOR COMPLEX PROBLEMS IN BATTLEFIELD 
MANAGEMENT AND TACTICAL FUSION OF INTELLIGENCE, ETC. 

• TIME IS RIGHT FOR NASA TO TAKE ADVANTAGE OF THIS 
POWERFUL TECHNOLOGY. TAILORED NEUROPROCESSORS 
WILL BE IDEALLY SUITED TO SATISFY NASA'S UNIQUE AND 
GROWING DEMANDS IN COMPUTATION AND DATA 
MANAGEMENT WITH THE EVOLUTIONARY DEVELOPMENT OF 
SPACE STATION. 
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